Rapid Iterative Testing and Evaluation (RITE) Usability Testing
Background
As the Usability Researcher of UOB TMRW, I'm tasked with the experimentation of designs and setting up usability testing capability in Bangkok and Jakarta. With my understanding of UX research methods and armed with a strong arsenal of tools, there is particular method which had been used more than others.
In this post I would like to document my experience with the RITE usability testing method, the good and bad of it and what are the key things to lookout for when planning a usability testing using the RITE method.
What is RITE?
RITE stands for Rapid Iterative Testing and Evaluation, the method involves updating the prototype as the usability test is ongoing. When the workgroup and the researcher believes they have enough evidence through the usability test that a specific design is not working, they can make a decision to changeout a design for an alternative. With the new design alternative, they can then test it with the remaining participants.
Why use RITE?
RITE as the name suggest enables the team to quickly learn and re-test their design hypothesis. I will choose to use RITE testing method in the following scenario.
1. Research cost is high and research budget is low, this results in lower testing frequencies.
2. The project team runs in Sprint or have an Agile culture, design team is challenged to test and evaluate the usability of designs within a sprint or two.
When to use RITE?
Overseas Usability Research
Overseas testing are usually more costly and less frequent hence, businesses and project teams would usually utilise a single trip to test out multiple hypothesis.
RITE is a useful method as it optimise the limited time spent for research to test out alternative designs. Once a specific design had failed we can switch out the design for an alternative.
Tight Research Budget (Limited participants)
Similarly, where the constraint is budget and hence limited participants are recruited for testing. RITE can be employed to test and validate multiple design hypothesis with the limited number of participants. Typically I would run a full test with 10 participants broken into 2 sets of 5 each. For each design hypothesis failing 3 times, a change to the alternative will be recommended.
When not to use RITE?
While the RITE method is rather versatile and useful for Lean and Agile teams the following would not be appropriate to employ RITE as a testing method.
1. When you are doing a quantitative study. This point is rather straight-forward, RITE is a qualitative testing method which gives you a general observation of the primary design and its alternative, no statistical inference can be made from RITE method.
2. When your hypothesis is to compare 2 designs and find out which will convert users better. Although this may look similar to what we are doing with RITE but it is a very different hypothesis, in this instance we have two variants and we want to know with confidence which design have a better chance of conversion and maybe the Confidence Interval (CI) between this 2 variant. For this case, the appropriate method to use will be A/B testing.
How to run RITE?
The approach to running RITE is similar to running any usability test where you will prepare the interactive prototype, test script and participant criteria. Preparation of all these material will usually take up 2 weeks, the actual testing may take between 4 to 5 days.
The approach is slightly different between RITE and the usual usability testing. For the usual usability test, the researcher will follow the test script and run the course of the test for the total number of participants. In RITE, the total participants will be broken into 2 groups and 1 group will run after another. Test script and the prototype may differ between group 1 and 2.
Let's look at a simple illustration of how a RITE test may be carried out. For this purpose, I am running a usability test of a feature and I have 4 days to administer this test locally. The following will illustrate how this 4 days look like for a RITE test.
Day 1 - Preparation and setup
- Gather the team and initiate the testing through alignment of roles and expectations (Researchers, observers, note-takers, etc.)
- Setup and introduce the space (Holding space, testing room and online/offline observers room).
- Walkthrough events that will happen in the next 4 days, it is important to let them know that we will have time for feedback (daily) and also time to make changes on the prototype (Day 3 of testing).
Day 2 - Test day 1
- Test day 1 will run with group 1 of your participants through the unaltered interactive prototype.
- Feedback and observations will be gathered at the end of the last participant.
- Observers are encouraged to note down questions they may have and an online chatroom will be provided between the testing room and observation room for questions to the participants.
Day 3 - Iteration day
- With the feedback from the first group of participants, the UX researcher will brief the product managers/owners, the designers and engineers on possible problem in the design and flow based on observations. The team will then make a decision on the changes to make on the prototype and test script or bring forward the alternative solution which they would like to test.
Day 4 - Test day 2
- Testing with group will be administered with the alternative or enhanced solution.
- Feedback and observations will be shared at the end of the last participant.
- Research and the working team will discuss the observations and recommendations.
That sums up the 4 days of RITE testing.
What is Key to running RITE successfully?
1. Establish a good observation room with the variants and screen flows map out, post-its to capture observation and feedback from everyone observing.
2. Involvement of the working team and stakeholders will help make an effective iteration for testing. One of the worst setback after a RITE is the successful option being either inviable or technically not feasible.
3. Define clearly what is a 'major concern', 'could be a concern' and 'negligible'. Do not define it too granular as there are not too many participants in a RITE testing. (E.g. with n=5, a total of 4-5 observations will deem a task 'major concern', 2-3 observations 'could be a concern', 1 observations 'negligible').
4. Focus iterating effort on tasks that is of major concern, task that could be a concern can be further monitored with analytic tools (e.g. Hotjar, UXCam, etc..) or tested remotely (e.g. Usertesting.com, Maze, Userlytics, etc..).
Conclusion
RITE usability testing method had been a very useful and versatile research method as more teams are moving into Lean and Agile, where fail and learn fast is key. RITE provides you a structure to test, fail and relearn again through an iterative approach.
